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DECLARATION OF DR. NIALL R. LYNAM UNDER 37 CFR §1.132 
I, Niall R. Lynam, Ph.D., do hereby declare and state: 

1 . I am the Senior Vice President and Chief Technical Officer of Magna Mirrors of 
America, Inc. (a.k.a. Magna Donnelly Corporation, a.k.a. Donnelly Corporation, 49 West 3 rd 
Street, Holland, Michigan 49423), and I have been with the company since the early 1980s. My 
responsibilities at Magna Mirrors/Magna Donnelly/Donnelly Corporation (hereafter "Donnelly") 
have included research and development of new products, including vehicular vision systems as 
well as technology and business management. I submit this declaration in relation to U.S. patent 
application Serial No. 10/643,602, invented by Schofield et. al ("Schofield"). I submit this 
declaration in response to, and necessitated by, the Office Action mailed by the United States 
Patent and Trademark Office ("USPTO") on February 24, 2010. 

2. I make this declaration of my own knowledge and could and would testify 
competently as to the matters set forth herein, if called upon to do so by a court of law. 

3. I graduated in 1975 from the University College Dublin receiving a Bachelor of 
Science degree in chemistry and mathematics and remained at the University College Dublin as a 
postgraduate student and was conferred with a Ph.D. in 1981. My doctoral thesis was on an 
opto-electronic topic and included optics and electronics developments. 

4. I joined Donnelly Mirrors Ltd. (a subsidiary company of Donnelly) in 1980 as a 
research scientist. 
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5. In 1984, 1 transferred to Donnelly Corporation as a principal scientist in Applied 
Research. In 1985, 1 was appointed manager of Applied Research and, in 1986, 1 joined 
Donnelly's Corporate Management Team. In 1992, 1 was appointed Vice President, Corporate 
R&D. In 1995, 1 was appointed Senior Vice President, Chief Technical Officer. Throughout my 
career with Donnelly, I have been involved in the development of new products including 
vehicular vision systems and other optical, electronic and mechanical products for various areas 
of an automobile. 

6. I hold numerous patents in the field of automotive products, several of which are 
related to vehicular vision systems, including United States Patent Nos. 4,712,879; 5,122,647; 
5,285,060; 5,715,093; 5,929,786; 6,222,460; 6,353,392; 6,445,287; 6,534,884; 6,690,268; 
6,806,452; 7,168,830 and 7,224,324. 

7. The '602 patent application is one of several patent applications and patents 
relating to automotive vision systems owned by Donnelly. 

8. I have reviewed the '602 patent application, and I have reviewed its prosecution 
history at the USPTO and its associated prior art. I have also reviewed the papers and art 
presented and considered in the present examination of the '602 patent application, including the 
Office Action dated February 24, 2010 and the references cited in the Office Action. 

9. Prior to and contemporaneous with the priority filing to parent application, which 
issued as U.S. Patent No. 5,670,935 (based on U.S. patent application Serial No. 08/445,527, 
filed on May 22, 1995), I was responsible for Donnelly's Applied Research Department. Since 
the filing date, I have continued to be familiar with vehicular vision systems in general, and I am 
familiar with the commercial usage of vision systems on vehicles. 

10. I am familiar with the state of the art as of the early '90s in the arena of vehicular 
vision systems in general and vehicular camera-based back-up systems in particular. I am also 
familiar with the extensive experimentation and testing undertaken by Schofield et al. and 
required to develop the inventions that led to the '602 patent application. I concur with 
Schofield's disclosure in the Background to the Invention of the '602 patent application that, as of 
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the early '90s, camera-based rearview vision systems for vehicles had not obtained commercial 
acceptance on the likes of the vehicles illustrated in the drawings. 

11. In my opinion, a person that is of ordinary skill in the art of the subject matter of 
the Schofield Vision System would have a Bachelor's Degree in electrical engineering, or a 
Bachelor's Degree in mechanical engineering with experience in cameras and image processing 
and displays, or a Bachelor's Degree in science with experience in physics, or equivalent thereof. 
One of ordinary skill in the art would have at least two years of experience in the field of the 
subject matter, such as two or more years of experience in the field of automobile imaging 
systems or vision systems and/or automobile monitoring systems. One of ordinary skill in the art 
would be familiar with image capture devices or cameras and displays and associated parts and 
operation, including construction and sources of supply thereof. For example, one of ordinary 
skill in the art would be an engineer or technical person working at an automobile manufacturer 
or at a supplier of vision systems to an automobile manufacturer or comparable entity. At the 
time of the Schofield et al. invention, I was at least of ordinary skill in the art of the subject 
matter of the Schofield Vision System. 

12. I understand that my Declaration will accompany Donnelly's response to the 
Office Action in the examination of the '602 patent application. I have reviewed that response in 
preparing my Declaration. 

13. The claimed inventions of the '602 application encompass a vehicular vision 
system, including as exemplified by claims 50, 92, 99, 102 and 105: 

(i) a vehicle equipped with two or more image capture devices, said two image 
capture devices capturing images external of the vehicle, said two image capture 
devices having overlapping fields of view, (ii) said vehicle equipped with an image 
processor, wherein image data captured by said two image capture devices are 
processed by said image processor, (iii) said image processor producing a 
synthesized image from said image data captured by said two image capture 
devices, (iv) said synthesized image comprises a composite image of said image 
data captured by said two image capture devices without duplication of image 
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information, (v) said vehicle equipped with a display screen displaying said 
synthesized image, (vi) said synthesized image displayed as a single image on a 
single display screen that is viewable by a driver of said vehicle when the driver is 
normally operating said vehicle, (vii) wherein the displayed image displayed on 
said single display screen includes an image portion from an image captured by 
each of said two image capture devices. 

I will hereafter in this Declaration refer to (i) through (vii) above collectively and combined 
together as the "Schofield Vision System." 

14. Exhibit A attached hereto is a 1999 Society of Automotive Engineers (SAE) paper 
titled "Panoramic Electronic Rear Vision for Automotive Applications" and authored by Rich 
Hicks, Ken Schofield, Paul Tarno and Mike Veiseh of Donnelly Electronics. This SAE paper 
was presented at the International Congress and Exposition in Detroit, Michigan on March 1-4, 
1999. The SAE International Congress and Exposition is a prestigious international Congress 



and Exposition and technical 
papers such as in Exhibit A are 
invited for presentation at 
Technical Sessions attended by 
engineers and interested persons 
drawn from the global automotive 
technical and business 
community. Page 3 of Exhibit A 
is reproduced to the right. 
This SAE paper (Exhibit A) is co- 
authored by Ken Schofield, one of 
the inventors of the '602 
application, and shows how a 
displayed image comprises a 
synthesized image that provides a 
virtual image that shows the entire 
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scene rearward of the vehicle. As can be seen above, the synthesized image comprises a 
composite image of image data captured by two or more image capture devices without 
duplication of image information, and the synthesized image is displayed as a single image on a 
single display screen (see Merged Image of Figure 2 above), and the displayed image displayed 
on the single display screen includes an image portion from an image captured by each of the 
image capture devices (see the Left Camera Image, Center Camera Image and Right Camera 
Image of Figure 1 above). When the two or three images are merged and the synthesized merged 
image is displayed on the single display screen, the displayed image approximates a rearward 
view from a single location and may provide a substantially seamless view rearwardly of the 
vehicle. 

15. Exhibit B attached hereto is a Ward's Auto World article dated May 1, 1998. 
Ward's Auto World is a well-known and widely accessed automotive publication with widespread 
circulation throughout the global automotive technical and business community. Exhibit B 
discusses, at page 2, Donnelly's development and production of a three camera vision system, 
which as two side cameras and a third rear camera, with the images captured by the cameras 
converged into one, which can be displayed for the driver on a screen on the instrument panel. 
The article states that "Combining the images into one central location could be an 
improvement." 

16. Exhibit C attached hereto is a Donnelly 2000 Company Profile. This publication 
was broadly circulated by Donnelly around 2000 to interested members of the automotive 
technical and business community. Exhibit C discusses the GM Precept concept vehicle 
introduced in late 1999, which included a three camera vision system of the type disclosed and 
claimed in the '602 application. Exhibit C, at pages 3 and 4, states that "Donnelly's unique 
system is built around proprietary software that merges the digital images from three or more 
separate cameras into a single panoramic image. Through that image, drivers have a clear, 
unified view of the rear and sides of the vehicle." 

17. Exhibit D attached hereto is a GM Opel Press Release dated January 5, 2001 . 
This is a press release released by GM's Opel division. Exhibit D also discusses the GM Precept 
concept vehicle and Donnelly's vision system (which corresponds to the Schofield Vision 



Applicants : Kenneth Schofield, Mark L. Larson and Keith J. Vadas 
Serial No. : 10/643,602 
Page 6 of 21 



System disclosed and claimed in the '602 patent application). Exhibit D, at page 3, states: "So 
efficient is the Precept's shape that just the addition of conventional outside rear-view mirrors 
would increase drag by more than 17 percent. Consequently, the Precept employs two tiny 
cameras instead of outside rear-view mirrors which, when combined with a third rear-facing 
camera located inside the rear window, produce an integrated panoramic rear view on a 
reconfigurable LCD display easily viewed by the driver." 

18. Exhibit E attached hereto is an article from www.theautochannel.com that touts 
the Donnelly PanoramicVision system on a Renault Talisman concept car at the 2001 Frankfurt 
Motor Show. This publication was broadly circulated and accessed by interested members of the 
global automotive technical and business community. Exhibit E states: "Donnelly's next 
generation PanoramicVision™ system is demonstrated on the Renault "Talisman" concept car. 
There are no mirrors on the car. Instead, this system digitally merges the images from three 
cameras to relay a seamless panoramic view of the area behind and around the vehicle. The view 
is displayed on a screen located on the upper part of the dashboard. The screen also provides 
information on navigation, the sound system, the heating and cooling system, and the vehicle's 
warning and security system." 

19. Exhibits A-E are evidence of the pioneering development and commercial success 
of the claimed Schofield Vision System invention of the '602 application, as well as the praise 
and accolades it received when disclosed by leading automakers at prestigious international 
automotive shows on various concept cars. Thus, these Exhibits evidence the non-obviousness 
of the Schofield Vision System at least because, about 5 years or so after the filing of the parent 
of the '602 application, experts and automakers experienced in the field were sufficiently 
impressed with the Donnelly vision system that they included it in their highly anticipated 
concept vehicles, and experts and media pundits touted its uniqueness and the improvements and 
enhancements it would provide to vehicles. The commercialization and production of the 
Donnelly vision system, such as in the Precept and Talisman vehicles in 2000 and 2001, clearly 
evidence the commercial success of the Schofield Vision System. And the fact that, several 
years after the filing of the parent Schofield priority patent application, experts were still quick to 
praise Donnelly's vision system as pioneering and unique, which further evidences the 
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commercial success of the Schofield Vision System and the failure of others to invent and 
produce a vision system suitable for use in vehicles that would provide a single display of an 
image synthesized from multiple images captured by multiple cameras of the vehicle. The 
articles and publications of Exhibits A-E thus evidence the non-obviousness of the claimed 
invention of the '602 application. 

20. The teachings and claimed subject matter of the '602 application have been 
important to the commercial success of such camera-based vision / display systems. In my 
opinion, Scho field's innovation of having the multiple images merged and a single image 
synthesized for display on a single display screen collectively and in combination with the other 
elements of the claimed invention provides an effective solution to the longstanding problem of 
providing information to a driver in a meaningful manner that does not distract the driver when 
he or she is driving the vehicle, such as during a reversing maneuver. 

21 . There is abundant evidence, such as is shown in Exhibits A-E, that the Schofield 
claimed invention of the '602 application was not obvious to one skilled in the art as of May 22, 
1995. Such evidence includes the successful use of the synthesized image and single display 
screen with the multiple elements of the Schofield Vision System for the likes of the GM Precept 
concept vehicle and the Renault Talisman vehicle commercially successfully supplied by 
Donnelly. The Schofield Vision System, as claimed in the '602 application, satisfied a long-felt 
need. And the non-obviousness of the Schofield '602 claimed inventions is further evidenced by 
Donnelly's use on such vehicles. 

22. When Renault, following the debut of the Precept and after seeing the accolades 
given to Donnelly's vision system, wanted to provide a rear vision system in its Talisman 
concept vehicle to be shown at the prestigious Frankfurt Motor Show, it was Renault's desire to 
have the proven performance and benefits of the Schofield Vision System invention (and not 
merely some increasing popularity of a particular vehicle model or the effectiveness of the 
marketing efforts or advertising employed) that drove to Donnelly's commercial success in being 
able to work with Renault and provide the vision system to Renault to begin with and that drove 
to Donnelly's commercial success in the selection of the Schofield Vision System for the Precept 
and Talisman vehicles. In terms of business awards by automakers, advertising or the like by 
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suppliers do not typically play a role in winning business, and thus the use of the Schofield 
vision system on the Precept and Talisman vehicles is based on the technical merits of the 
claimed Schofield vision system invention. The OEM market and supplier market is relatively 
small and extraordinarily competitive - so neither advertising, ties to the supplier, nor other 
similar business reasons have bearing on the choice of rear vision system for such applications. 

23. While there may be many reasons a consumer may choose to buy a vehicle at a 
dealership, there is a reason that Renault chose to have the Schofield Vision System in its 
Talisman vehicles — and that reason is that by the time Renault began to develop the Talisman 
vehicle, the Schofield Vision System had already been a proven success on the prior GM Precept 
vehicle. 

24. The commercial success achieved by Donnelly with the Renault Talisman 
vehicles that use the Schofield Vision System is not the result of heavy promotion or advertising, 
or a shift in advertising, and is not the result of consumption by purchasers tied to the assignee of 
the present application (neither General Motors nor Renault is tied to Donnelly and neither 
General Motors nor Renault has a vested interest in the outcome of this patent prosecution), and 
is not the result of other business events beyond Renault's desire to use the proven Schofield 
Vision System. Renault wanted the proven success and reliability and performance of the rear 
vision system of the Renault Talisman vehicles to match that of the proven prior GM Precept 
rear vision system, and thus chose the Schofield Vision System for its Talisman vehicle. 

25. It is my opinion that the use of the Schofield Vision System by the likes of 
General Motors and Renault is evidence of the commercial success of the presently claimed 
invention. The Schofield vision system was shown at prestigious events and the shown vision 
system utilized the Schofield Vision System, with the claimed elements of the claimed system 
being important elements in the success of the concept vehicles shown. To have the so selected 
and so used Schofield Vision System shown at such prestigious events and implemented as part 
of new pioneering vehicles, clearly evidences the pioneering and novel and non-obvious aspects 
of the Schofield Vision System. And the praise and accolades given by the experts (see Exhibits 
B, D and E attached hereto) upon review of the Schofield Vision System is further evidence of 
the pioneering and novel and non-obvious aspects of the Schofield Vision System. 
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26. It is my opinion that the high praise and accolades given to the Schofield Vision 
System is in and of itself strong evidence of the unexpected results of the vision system that is 
disclosed and claimed in the '602 patent application. 

27. If anything, in my opinion, the other vehicle vision systems disclosed in the likes 
of the applied art of Secor and Nishimura and Tuck, constitute failed attempts by others to 
conceive and develop a rear vision system that would provide a display of a rearward view in a 
meaningful manner that would not distract the driver while the driver is operating the vehicle, 
such as during a reversing maneuver, and in accordance with the claimed subject matter. 
Otherwise, and logically, it would have been the likes of the applied art of Secor, Nishimura 
and/or Tuck that would have been used by leading automakers for their concept vehicles, and 
such would have occurred (but did not occur) years earlier than the filing date of the Schofield 
priority application. 

28. Further, it is my opinion that the use of the Schofield Vision System on the likes 
of the concept Precept and Talisman vehicles, several years after the initial filing of the priority 
application, itself evidences the skepticism of experts for what went before, and thus reinforces 
the innovativeness of the claimed Schofield Vision System. 

29. It is my opinion (as at least one of ordinary skill in the art at the time of the 
invention), that the abundance of secondary considerations evidenced above show that the 
combination of features that collectively and combined together constitute the Schofield '602 
application claims would not in 1995 have been predictable to one of ordinary skill in the art. 
Indeed, the commercial success of what Schofield invented evidences the uniqueness and 
innovativeness of the Schofield Vision System, such as described and claimed in the Schofield 
'602 patent application. 

30. The Office Action rejects the claims of the Schofield '602 application in view of 
various combinations of Secor, Fukuhara, Nishimura, Tuck and Kishi. It is my opinion (as at 
least one of ordinary skill in the art at the time of the invention) that Secor, either alone or in 
combination with Fukuhara or Nishimura or Tuck or Kishi or other prior art of record, does not 
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disclose or suggest or render obvious to one of ordinary skill in the art the claimed subject matter 
of the Schofield Vision System. 

31. I have reviewed Secor, U.S. Patent No. 5,289,321, and note that Secor discloses 
multiple cameras and multiple display screens, with each of the display screens displaying 
images from a respective camera. It is my opinion that Secor clearly teaches towards use of a 
display screen to display images from only a single camera at a time, and thus teaches away from 
displaying a synthesized image displayed on a single display screen that is viewable by a driver 
of said vehicle when the driver is normally operating said vehicle, with the synthesized image 
being produced by an image processor from image data captured by two image capture devices, 
with the synthesized image comprising a composite image of the image data captured by the two 
image capture devices without duplication of image information, and with the displayed image 
displayed on the single display screen including an image portion from an image captured by 
each of the two image capture devices, such as is clearly included in the Schofield Vision 
System. 

32. Clearly and unequivocally, Secor distinguishes its side- 

rear view camera 22 from its side-looking camera 34. This is plain ^^^^r^. 



reproduced to the right), and it is also plain and unequivocal from the 
description of Secor and how Secor describes and distinguishes the 
function of camera 22 from the function of camera 34. Secor also 

discloses that the display screen 42 can be switched between the side-rear view camera assembly 
22 and the side-looking camera assembly 34 (see column 5, lines 34-41 of Secor). For example, 
and as clearly disclosed in Secor, the "switching of the screens back from the camera assemblies 
34 to the side camera assemblies 20 and 22 occurs automatically, for example, if the vehicle is 
shifted out of reverse into a forward speed" (see column 5, lines 37-41 of Secor). Thus, Secor's 
display screen 42 only discloses images from one of the cameras at a given time, and the images 
captured from either the side-rear view camera (22) or the side-looking camera (34) are shown 
on the display screen (42), but the system of Secor never shows images from both cameras at the 
same time on the same screen. 



from simply looking at the Figures of Secor (such as Figure 3, 
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33. I have reviewed Nishimura, U.S. Patent No. 4,713,685, and note that Nishimura 
discloses use of a wide display for displaying two separate, distinct and isolated images. The 
images are captured from opposite sideward facing cameras (see column 4, lines 36-41 of 
Nishimura). The separate images are played on both sides of the display, which is divided by a 
vertical center line (see column 2, lines 26-29 of Nishimura). 

To further separate and isolate the separate images from one d :•;:::■> U ? i^;;i^|*^?!r| 

another, a switch is responsive to a turn signal of the vehicle 4?^ r^'-rH;„;:;::_j : ^....^^S 

being operated and issues a signal for displaying a flashing sign f^TV* 
on the frame part (80) of the display (78) at the respective l^^j 
separate image (72, 73). See column 7, lines 6-28, and Figure 7 j ""{ f * [ 

of Nishimura, reproduced to the right. u l 

34. Thus, it is my opinion that Nishimura does not disclose or suggest a display 
screen displaying a synthesized image, with the synthesized image comprising a composite 
image of image data captured by two or more image capture devices without duplication of 
image information. To the contrary, Nishimura discloses two separate images of areas at 
opposite sides of the vehicle, with the displayed images providing distinct and separate views of 
distinct and separate regions exterior the vehicle, and with the displayed images clearly 
providing respective views from separate and distinct respective locations. It is my opinion that 
Nishimura thus teaches away from providing a synthesized image and display of the Schofield 
Vision System by Nishimura's teaching towards use of a wide display for displaying to separate, 
distinct, non-joined, isolated images in a side-by-side and separated and demarcated manner. 
Such a separate display of images is vastly different from the synthesized image that comprises 
merged together images captured by cameras with overlapping views to provide the driver with a 
single image of the area rearward of the vehicle, such as is clearly disclosed and claimed in the 
Schofield '602 application. 

35. I have reviewed Fukuhara, U.S. Patent No. 4,653,316, and note that Fukuhara 
discloses an apparatus for detecting and analyzing cracks in a road surface, with a laser beam 
scanning apparatus for detecting road surface conditions, and with no display of captured video 
images. As disclosed in Fukuhara, the laser beam scanning apparatus includes two television 
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cameras that are directed at a 60 degree downward angle for catching the locus of the laser beam on 
the road surface and reflected light. The cameras have non-overlapping fields of view such that the 
cameras 22, 23 pick up the scanning loci 40 of the laser beam respectively in the range of positions 
Pi - P 2 o48 and P2049 - P4096 (see column 3, lines 56-60 and Figure 6 of Fukuhara, reproduced below). 



The cameras 22, 23 of Fukuhara capture the scanning locus of the laser beam for producing a 
transverse profile data of the road surface. This is shown in Figure 7 of Fukuhara (reproduced 
above), where the camera 22 (23) captures multiple transverse road profiles as the laser is 
scanned across the road surface in front of the camera (column 3, line 61 to column 4, line 7 of 
Fukuhara). The road profile data of the respective cameras are synthesized into a picture image 
representing the cross-sectional profile of the road (column 4, lines 22-24 of Fukuhara). The 
data regarding the transverse profile, cracks and the longitudinal profile are recorded on the VTR 
(column 6, lines 3-5 of Fukuhara), and "the data stored in the memory devices is converted into 
binary values in the processor 40 in accordance with a threshold value Ls shown in FIG. 13 so 
that the position of the crack is judged in accordance with data less than the threshold value Lc." 
(column 6, lines 16-20 of Fukuhara). "The result of judgment is displayed on display means, not 
shown such as a cathode ray tube" (column 6, lines 20-22 of Fukuhara). 

36. The Office Action dated February 24, 2010 in the Schofield '602 patent 
application, at page 3, states: 
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Fukuhara discloses *an image processor producing a synthesized image from the 
outputs of the image capture devices by at IbbbX ona of; fuminant biertdlng, chromlnant 
blending, dynamic range extending, pixe! group compensation, anthbjoO'm3ng: f multiple 
exposure, image morphing compensation, or image warping compensation" {Fukuhara: 
figures 4 and 8; column 4 f lines 13-28), 



I disagree. I have reviewed Fukuhara and Fukuhara does not disclose such an image processor 
or such image processing techniques. Column 4, lines 13-26 of Fukuhara states: 

FIG. 4, the data read out from respective television 
cameras are applied to the synthesizing circuit 33, 
where the data are synthesized m a manner to be de- 
scribed later and then recorded on a VTR 3& The 
image pick-up surface has a performance of temporarily 
storing arid integrating $€> that picture image data writ- 
ten on the picture pick-up surface one frame before are 
read out. The reading out of the data is performed in 
synchronism with the output signal from the optical 
sensor 19, 

The synthesizing circuit 33 synthesizes picture im- 
ages read out from respective cameras 22 an4 23 into a 
picture image representing the cross-sectional profile 
FIG. 8 shows the manner of the synthesis. 

Thus, Fukuhara merely discloses synthesizing circuit that synthesizes data that are recorded on a 
VTR. There is no disclosure or suggestion in Fukuhara of an image processor producing a 
synthesized image from image data captured by two image capture devices with overlapping 
fields of view, with the synthesized image comprising a composite image of the image data 
captured by the two image capture devices without duplication of image information, and with a 
display screen displaying the synthesized image as a single image on a single display screen that 
is viewable by a driver of the vehicle when the driver is normally operating the vehicle, and with 
the displayed image displayed on the single display screen including an image portion from an 
image captured by each of the two image capture devices, such as included in the Schofield 
Vision System. Nor is there a disclosure or suggestion in Fukuhara of an image processor 
processing image data captured by image capture devices by at least one technique chosen from 
luminant blending, chrominant blending, dynamic range extending, pixel group compensation, 
anti-blooming, multiple exposure, image morphing compensation and image warping 
compensation, such as is also included in claims 50 and 109 of the '602 patent application. 
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37. It is my opinion that there is no disclosure or suggestion in Fukuhara as to where 
this display means is located (it may be remote from the vehicle for later analysis of the 
processed data), and clearly there is no disclosure in Fukuhara that the display means displays 
images, since the display means is clearly for displaying the result of the judgment of the 
position of the crack. Thus, Fukuhara displays a result of judgment of the position of the crack 
based on processing of binary values of the data stored in the memory devices. There is 
absolutely no disclosure or suggestion in Fukuhara of displaying images on a video display 
screen, and particularly there clearly is no disclosure or suggestion in Fukuhara of a synthesized 
image displayed as a single image on a single display screen that is viewable by a driver of the 
vehicle when the driver is normally operating the vehicle. Fukuhara (at column 4, lines 13-26) 
merely states that "the data read out from respective television cameras are applied to the 
synthesizing circuit 33, where the data are synthesized in a manner to be described later and then 
recorded on a VTR 35." Fukuhara is wholly silent as to a disclosure or suggestion of an image 
processor that processes image data captured by the image capture devices by at least one 
technique chosen from luminant blending, chrominant blending, dynamic range extending, pixel 
group compensation, anti-blooming, multiple exposure, image morphing compensation or image 
warping compensation. Such processing clearly is not described in Fukuhara. 

38. Fukuhara does not disclose or suggest such a vision system, but rather discloses a 
laser beam scanning apparatus for detecting road surface conditions. Fukuhara does not disclose or 
suggest a display of images on a display screen that is viewable by a driver of the vehicle when the 
driver is normally operating the vehicle. To the contrary, the television cameras of Fukuhara 
catch the locus of a laser beam on the road surface to determine the cross-sectional profile of the 
road surface. The data regarding the transverse profile are recorded on a VTR and the stored data 
is converted into binary values in the processor in accordance with a threshold value so that the 
position of a crack is judged in accordance with data that is less than a threshold value. Fukuhara 
discloses that the result of the judgment (of the position of the crack) is displayed on display means 
(see column 6, lines 3-23 of Fukuhara). Thus, nowhere in Fukuhara is there a disclosure or 
suggestion that images are captured and displayed to the driver of vehicle, and clearly there is no 
disclosure or suggestion in Fukuhara that a synthesized image (comprising a composite image of 
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merged image data captured by two image capture devices without duplication of image 
information in the manner claimed in the Schofield '602 application) is displayed on a single 
display screen that is viewable by a driver of the vehicle when the driver is normally operating the 
vehicle, with the displayed image displayed on a single display screen and including an image 
portion from an image captured by each of the two image capture devices, and nowhere is there a 
disclosure or suggestion of an image processor that processes image data captured by the image 
capture devices by at least one technique chosen from luminant blending, chrominant blending, 
dynamic range extending, pixel group compensation, anti-blooming, multiple exposure, image 
morphing compensation or image warping compensation, such as clearly disclosed and claimed 
in the Schofield '602 patent application. 

39. I have reviewed Tuck, U.S. Patent No. 4,772,942, and note that Tuck discloses a 
display system for a tank that includes individual display units mounted side-by-side, each having a 
respective television camera associated therewith so that a substantially continuous picture of at 
least part of the surrounding panorama can be reconstructed 
and viewed by an observer. See, for example, the Abstract 
and Figure 3 of Tuck, reproduced to the right. As can be 
seen in Figure 3 of Tuck, each camera 10, 1 1, 12, 13 is 
associated with a respective display generator 15, 16, 17, 18, 
which display the respective images to the observer "O" 
through respective magnifying lenses 20, 21, 22, 23. There 
is no disclosure or suggestion in Tuck of a display system that displays a composite image 
synthesized from image data captured by the image capture devices without duplication of image 
information, and with the display system displaying the composite image on a single display 
screen of the vehicle. Nor is there a disclosure or suggestion in Tuck of the display system 
displaying a displayed image on a single display screen that includes an image portion from an 
image captured by each of the image capture devices or cameras. Moreover, it is my opinion 
that Tuck would teach one of ordinary skill in the art away from such a vision system by its 
teaching that each of the individual display units has a respective television camera associated 
therewith, such that images captured by each television camera are displayed on their associated 
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display unit such that the displayed image on one of the individual display units cannot have an 
image portion from an image captured by each of the image capture devices. 

40. Thus, it is my opinion (as at least one of ordinary skill in the art at the time of the 
invention) that none of the cited references discloses or suggests a single display screen displaying 
a synthesized image, with the synthesized image comprising a composite image of image data 
captured by two or more image capture devices without duplication of image information, and 
with the displayed synthesized image including an image portion from an image captured by 
each of two image capture devices having overlapping views, such as clearly included in the 
Schofield Vision System. As shown in Exhibit A (and see paragraph 14 above), such a displayed 
synthesized image may provide a substantially seamless view exterior or rearwardly of the 
vehicle, with the displayed image approximating a view from a single location. 

41 . In the Office Action dated February 24, 2010, the Examiner states that it would have 
been obvious to combine Secor with Fukuhara and/or Nishimura and/or Tuck to arrive at the 
claimed invention of the '602 application. I disagree. None of the cited references disclose or 
suggest a synthesized image as claimed. Secor discloses a display screen that displays images 
from only a single camera at a time, while Nishimura discloses a wide screen that displays 
separate images thereon, and Fukuhara does not disclose a display for viewing by the driver of 
the vehicle while the driver is driving the vehicle. It is my opinion (as at least one of ordinary 
skill in the art at the time of the invention), that, were one of ordinary skill in the art to look to 
the cited references (without also referencing the disclosure of the Schofield '602 application), 
one would not be led to the Schofield Vision System as described and claimed in the Schofield 
'602 application, but would rather be led to providing a display system for displaying separate 
individual images from one or more cameras with no synthesizing or merging of images captured 
by the one or more cameras of the vehicle. 

42. In my opinion (as at least one of ordinary skill in the art at the time of the 
invention), neither Secor nor Fukuhara nor Nishimura nor Tuck, either alone or in combination 
with one another or other of the cited prior art, discloses, suggests or renders obvious Schofield's 
claimed subject matter relating to an image processor producing a synthesized image from image 
data captured by two image capture devices with overlapping fields of view, with the synthesized 
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image comprising a composite image of the image data captured by the two image capture 
devices without duplication of image information, and with a display screen for displaying the 
synthesized image, and with the synthesized image displayed as a single image on a single 
display screen that is viewable by a driver of said vehicle when the driver is normally operating 
said vehicle, with the displayed synthesized image that is displayed on the single display screen 
including an image portion from an image captured by each of the two image capture devices. 
As discussed above, such a displayed synthesized image may provide a substantially seamless 
view rearwardly of the vehicle that approximates a view from a single location. Such a 
synthesized image and display are not disclosed or suggested in the references applied by the 
Examiner in the Office Action or other prior art of record. 

43. In my opinion, the Schofield '602 application is a pioneering and technically 
comprehensive teaching in its field. Within its eleven columns of specification and eleven 
figures (all filed in May 1995) are to be found many of the features of commercial multi-camera 
systems with graphic overlays only now, over a decade later, entering the market. Taking as an 
illustrative example the Donnelly PanoramicVision™ system and display, Schofield teaches use 
of such image synthesizing and merging to provide an exterior view to the driver of the vehicle at 
a single display in the vehicle. In my opinion (as at least one of ordinary skill in the art at the 
time of the invention), that this feature is only relatively recently being used on concept vehicles, 
such as is shown in Exhibits C, D and E, further evidences that the Schofield '602 application is a 
pioneering disclosure in this field. 

44. Regarding the Examiner's rejection in the Office Action mailed by the USPTO on 
February 24, 2010 of independent claims 50, 92, 99, 102 and 105, 1 respectfully traverse at least 
because Secor, Fukuhara, Nishimura and Tuck (taken alone or in combination) are utterly devoid 
of a disclosure or suggestion that their systems have a display screen that displays a synthesized 
image that is synthesized from image data captured by two image capture devices with 
overlapping fields of view, with the synthesized image comprising a composite image of the 
image data without duplication of image information. Nor do these references disclose or 
suggest such a synthesized image displayed as a single image on a single display screen that is 
viewable by a driver of said vehicle when the driver is normally operating said vehicle, with the 
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displayed synthesized image that is displayed on the single display screen including an image 
portion from an image captured by each of the two image capture devices. As discussed above, 
such a displayed synthesized image may provide a substantially seamless view exterior of the 
vehicle and may approximate a view from a single location. 

45. As one of at least ordinary skill in the art, it is my opinion that Secor, either alone 
or in combination with Fukuhara, Nishimura, Tuck and/or Kishi, fails to disclose or suggest to 
one of ordinary skill in the art at the time of the invention, the Schofield Vision System, as 
claimed in the '602 patent application. For example, none of the applied references disclose or 
suggest an image processor, wherein image data captured by image capture devices having 
overlapping fields of view are processed by the image processor, with the image processor 
producing a synthesized image from the image data captured by the image capture devices. Nor 
do the applied references disclose or suggest, for example, such a synthesized image comprising 
a composite image of the image data captured by the image capture devices without duplication 
of image information. Nor do the applied references disclose or suggest, for example, a display 
screen displaying such a synthesized image. Nor do the applied references disclose or suggest, 
for example, such a synthesized image displayed as a single image on a single display screen that 
is viewable by a driver of the vehicle when the driver is normally operating the vehicle. Nor do 
the applied references disclose or suggest, for example, such a displayed image displayed on 
such a single display screen including an image portion from an image captured by each of the 
image capture devices. With respect to the rejection of independent claims 92, 99 and 102, none 
of the applied references disclose or suggest a three camera system having a center image 
capture device with a field of view that overlaps the fields of view of each of two opposite side 
image capture devices, and with a display system that displays a composite image synthesized 
from image data captured by the image capture devices without duplication of image 
information, and with the displayed image including an image portion from an image captured 
by each of the side and center image capture devices. Thus, it is my opinion (as at least one of 
ordinary skill in the art at the time of the invention) that the Schofield Vision System, as 
disclosed in the Schofield '602 patent application and exemplified in the claims of the Schofield 
'602 patent application, is not disclosed or suggested or rendered obvious by the combination of 
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references applied in the Office Action dated February 24, 2010 in connection with the Schofield 
'602 patent application. 

46. In my opinion (as at least one of ordinary skill in the art at the time of the 
invention), that the results that would have been expected from the combinations of references 
applied in the Office Action dated February 24, 2010 would be to have a display means that 
displays judgment or analysis information, such as taught by Fukuhara (not images for viewing 
by a driver of the vehicle while operating the vehicle), or to have a separate display for each 
respective camera (such as taught by Secor and/or Tuck) or a display that switches between 
cameras so that an image from only one camera is displayed on the display at a given time (such 
as taught by Secor), or to have two separate and distinct images (and not a synthesized composite 
image) displayed side-by-side and isolated or separated on a wide screen to provide views of the 
opposite side regions exterior the vehicle on the wide screen (such as taught by Nishimura). It is 
my opinion (as at least one of ordinary skill in the art at the time of the invention) that it would 
clearly have been unexpected to one of ordinary skill in the art, armed with the proposed 
combination of applied references, to provide a single display screen for displaying an image 
synthesized from image data captured by two or more cameras, with the synthesized image 
comprising a composite image of the image data without duplication of image information, and 
with the displayed image displayed on the single display screen including an image portion from 
an image captured by each of the two image capture devices. Plainly, since an artisan armed 
with Secor would be switching the display between the cameras, it is my opinion (as at least one 
of ordinary skill in the art at the time of the invention) that there would not have been a 
motivation to combine the teachings of Secor with those of Nishimura (separate images on 
display) or Fukuhara (laser beam scanning apparatus for detecting road surface conditions, with no 
display of captured images). 

47. In summary, and as one who was active and of ordinary skill in this art back in the 
early to mid 1990s, it is my opinion, and as factually evidenced above, that the combination of 
features that collectively and combined together constitute the claims 50-52, 56, 58, 62, 67 and 
92-109 and new claims added in the Applicants' response to the Office Action in this 
examination of the Schofield '602 patent application is more than a predictable use of prior art 
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elements according to their established functions. For example, it is a fact as evidenced above, 
and is not speculation, that Secor clearly, literally and consistently discloses use of a display to 
display only one image from one camera at a time, whereas in sharp contrast, the claims of the 
Schofield '602 patent application include a synthesized image from multiple images captured 
from multiple image capture devices or cameras. Likewise, it is a fact as evidenced above, and is 
not speculation, that Nishimura clearly, literally and consistently discloses use of a wide display 
to display two separate, non-merged and non-synthesized images from opposite sideward 
viewing cameras, whereas in sharp contrast, the claims of the Schofield '602 patent application 
include a synthesized image from multiple images captured from multiple image capture devices 
or cameras having overlapping fields of view and with the images captured by the cameras 
merged in the claimed manner and without duplication of image information. Likewise, it is a 
fact as evidenced above, and is not speculation, that Fukuhara clearly, literally and consistently 
discloses use of an apparatus for detecting and analyzing cracks in a road surface, with a laser 
beam scanning apparatus for detecting road surface conditions, and with no display of captured 
images for viewing by the driver of the vehicle, whereas in sharp contrast, the claims of the 
Schofield '602 patent application include a synthesized image from multiple images captured 
from multiple image capture devices or cameras having overlapping fields of view and without 
duplication of image information, and with the images captured by the cameras merged in the 
claimed manner and displayed on a single display for viewing by the driver of the vehicle when 
the driver is normally operating the vehicle. Back in the early '90s, CMOS camera technology 
and corresponding image processing techniques were only emerging in automotive systems and 
Ken Schofield, one of the inventors of the Schofield '602 patent application, was recognized in 
the early '90s as a pioneer in the use of CMOS camera technology in automotive vision systems. 

48. The abundance of secondary considerations evidenced above, and such as is 
summarized above, show that the combination of features that collectively and combined 
together constitute the Schofield '602 patent application claims would not in 1995 have been 
predictable or obvious to one of ordinary skill in the art. In my opinion (as at least one of 
ordinary skill in the art at the time of the invention), the evidence presented above shows that one 
of ordinary skill in the art could not have arrived by known methods at the combination of 
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features that collectively and combined together constitute the Schofield Vision System as 
exemplified in the Schofield '602 patent application claims. The various elements in the various 
combinations of features that collectively and combined together constitute the Schofield Vision 
System do not merely perform the function that each element performs separately. This is 
evidenced such as by the incorporation of Schofield elements, and not those of Secor or 
Fukuhara or Nishimura or Tuck, into the likes of the camera-based vision systems shown in 
Exhibits C, D and E. Given that prior artisans (prior to Schofield et al. in 1995) tried and failed 
to provide a satisfactory camera system for providing a display of rearward information in a 
meaningful manner that did not distract from the driver's task of driving the vehicle, the 
commercial and unique success of the Schofield elements itself constitutes an unexpected result. 

49. All statements made herein of my own knowledge are true and all statements 
made on information and belief are believed to be true. I understand that willful false statements 
and the like are punishable by fine or imprisonment, or both, under 18 U.S. C. 1001, and that 
such willful false statements may jeopardize the validity of any patent that may issue from 
United States Patent Application Serial No. 10/643,602. 
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ABSTRACT 

Advances in electronics technology have opened numer- 
ous opportunities for new automotive products. One 
emerging area is the use camera systems to replace tra- 
ditional rear vision systems in automotive applications. 
This paper will discuss the advantages of such systems, 
what technical advancements have led to the feasibility of 
such systems, and an example implementation of an 
electronic rear vision system. 

INTRODUCTION 

Advances in camera, optics, and display technology have 
made it feasible to develop cost effective closed circuit TV 
systems for automotive applications. One function to 
which these systems may be applied is to supplement or 
replace existing, mirror-based, rear vision systems on 
vehicles. 

Replacing, or even supplementing, mirror-based rear 
vision systems should not be undertaken lightly The 
functionality and ergonomics of the rear vision system 
directly affect the safety and comfort of the driver, and a 
poorly implemented camera-based system may detract 
from both. On the other hand, a properly designed cam- 
era-based system can reduce driver workload and 
enhance safety. The key to this is providing an intuitive 
representation of the information available from the cam- 
eras that can allow drivers to be effortlessly aware of their 
surroundings. 

On the positive side, replacing mirrors with miniature 
cameras can have a positive impact on vehicle perfor- 
mance. Correctly implemented, camera systems can 
reduce the weight of the vehicle while reducing its aero- 
dynamic. This can result in significant Improvements to 
fuel efficiency. 

BACKGROUND 

In order to be successful, camera-based vision systems 
must significantly improve the driver's comfort and safety. 
While camera systems have several significant disadvan- 
tages when compared to traditional mirror-based sys- 
tems, camera systems offer tremendous flexibility in the 
presentation of the information to the driver. The key to 



success, therefore, is finding a presentation method that 
offers benefits that outweigh the system's [imitations. 

CAMERA SYSTEM FEATURES - Camera-based vision 
systems have a variety of features, some of which are 
advantageous and some of which are detrimental. The 
key to a successful system implementation is developing 
an architecture that utilizes the advantages of camera- 
based systems to more than compensate for the disad- 
vantages. The following sections describe several of the 
key features of a camera-based system and how they 
compare to mirror-based systems. 

Resolution -The resolution of a mirror-based system sig- 
nificantly exceeds the human eye's ability to resolve fea- 
tures and objects. The resolution of a camera-based 
system is, unfortunately, a cost versus performance 
trade. This trade affects the sensor, any processing elec- 
tronics and the display. While it may someday be feasible 
to provide a camera system with resolution comparable 
to a mirror system, current systems are severely limited 
in their performance. 

A successful implementation must, therefore, offer suffi- 
cient resolution for drivers to easily perform their task, 
while offering additional benefits to compensate for the 
loss of resolution. 

Field of View - The ability of a camera system to cover a 
wider and more complete field of view than a mirror sys- 
tem is one of its key advantages. While a mirror-based 
system is limited by obstructions between the driver and 
the rearward scene (vehicle pillars, passengers, etc} 
cameras can be placed on the exterior of the vehicle 
wherever necessary to cover the required field of view. A 
second limitation of a mirror system on the field of view is 
the amount of image distortion the driver can accommo- 
date. The larger the distortion, the larger the resulting 
field of view. Camera-based systems can either use mul- 
tiple cameras, or provide electronic distortion correction 
that can result in a large field of view without perceptible 
image distortion. 

Depth Perception Cues - Humans use a variety of cues 
to determine the relative distance between themselves 
and an object, The drivers will rely on several of these 
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cues at any time to make determinations regarding rela- 
tive and absolute distances between their vehicle and 
navigational obstacles. While the drivers can use any of 
these cues to some extent for any distance, they will tend 
to rely more on some cues than others depending on the 
distance to the object Some of these cues, and the 
ranges that they will tend to be most significant, are sum- 
marized in the following table: 



Table 1. Depth Perception Cues 



Cue 


Description 


Object 
Distance 


Stereoscopy 


The use of perspective 
differences between the 
images perceived by the 
left and right eyes to 
determine the distance 
between the observer and 
an object 


0-1 m 


Depth of 
focus 


The use of eye focal depth 
to determine the relative 
distance between two 
objects 


0-30 m 


Relative size 


The comparison of the 
perceived size of objects 
with aprlori knowledge of 
object to determine 
relative distance 


10m- 
Infinity 


Geometric 
perspective 


The comparison of relative 
motion between objects in 
the field of view 


10m- 
Infinity 



While driving, the operator is required to determine the 
relative distance between his vehicle and others within 
the range of 1 meter to 50 meters. In this range, the 
driver will rely on both depth of focus, relative size, and 
geometric perspective. While both mirror and camera 
systems provide relative size and geometric perspective 
cues, the camera system falls short on depth of focus. 

The mirror system simply folds the optical path of the 
image. This in essence maintains the optical distance, 
with some small error, between the driver and the object 
under observation. The camera system re-images the 
scene and provides the entire scene at a fixed focal 
depth that is determined by the display. The driver is, 
therefore, forced to rely solely on relative size to deter- 
mine distances. 

The camera system can, however, provide additional 
cues. Graphical depth cues can be added to the display 
to aid in the determination of relative distances. Lines 
can be overlaid on the camera scene indicating the dis- 
tance from the vehicle. Objects behind the cue are fur- 
ther from the vehicle than the calibrated distance while 
objects intersecting the cue are closer. Clever and intui- 
tive depth cues must be used in camera systems to com- 
pensate for the loss of depth of focus cues. 

I mage Pre s en tatio n - It is in the area of image presenta- 
tion that camera systems have significant advantages 



over mirror-based systems. Given the state of the art in 
today's processing technology, it is a straightforward mat- 
ter to warp, scale, rotate and translate camera images to 
whatever extent is necessary to provide an intuitive, eas- 
ily understood image to the driver. In addition, camera 
systems offer flexibility in the location of the information in 
the driver's field of view. Mirrors, by their very nature, dic- 
tate the location of the information. 

Night Performance - Night time driving represents the 
most challenging environment for rear vision systems. 
The system must cope with headlights that are extremely 
bright while presenting information on roads that are 
poorly lit In addition, the rear vision system has a poten- 
tial for interfering with the driver's forward field of view by 
introducing glare sources that can interfere with the 
driver's ability to adapt to a dark forward field of view. 

Mirror systems have traditionally only addressed night 
performance and glare sources with a simple prism 
mechanism on the interior mirror. It is only recently that 
mirrors have been introduced that can automatically 
darken to adapt to changing lighting conditions. These, 
electrochromic, mirrors have been applied to both interior 
and exterior rear view mirrors with good results. 

Unfortunately, the electrochromic mirror technology is 
limited in its response to quickly changing lighting condi- 
tions, in addition, the control mechanisms applied to 
these systems are limited in their complexity and the 
entire mirror is darkened uniformly. 

Camera systems have two advantages over electrochro- 
mic and prism technologies. The first is that the display 
luminance and image contrast are independent. The dis- 
play luminance can be set to match the luminance- of the 
drivers forward field of view regardless of the luminance 
of the rear scene. This results in a reduction of the 
driver's eye misadaptation that cannot be matched by a 
mirror system. 

The second advantage is that the image contrast may be 
manipulated differently for different regions of the image. 
Areas where the image is extremely bright can be 
dimmed, while areas that are poorly lit may be amplified. 
This can result in information being provided to the driver 
that is unavailable using mirror-based systems. 

SYSTEM ARCHITECTURE OPTIONS - Presenting 
information to the driver in the most intuitive fashion pos- 
sible is the primary consideration in defining the architec- 
ture of a camera-based rear vision system. The 
architecture must play on the strengths of the camera 
system sufficiently to more than compensate for the 
resulting loss of resolution and depth of focus cues. 
Three options are described in the following sections. 

Multiple Displays - The most straightforward approach to 
using cameras to replace mirrors is to simply place a 
camera at each outside mirror location and one on the 
rear of the vehicle and tie each camera video to a display. 
This is illustrated in Figure 1. This approach has been 
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demonstrated In a number of concept vehicles including 
several demonstrated at the 1998 Detroit Autoshow, 

The advantage of this approach is that it Is straightfor- 
ward in Its implementation and closely mimics the behav- 
ior of the traditional mirror systems with which roost 
drivers are already familiar. 

There are two major disadvantages to this approach. 
The first is that the most expensive component In the 
camera system is the display and this approach requires 
multiple displays. This can result in a system that is not 
particularly cost effective. 

The second disadvantage is that the information Is not 
presented in a particularly intuitive fashion. Drivers are 
required to scan multiple images that contain redundant 
information and integrate that Information in order to 
understand their surroundings. This provides little advan- 
tage to drivers In terms of ease of use and in fact may be 
detrimental to their performance as depth cues and reso- 
lution are reduced, 



Cameras, * * " 
Replacing Mirrors 





Left Center Right 
Camera Camera Camera 
Image Image Image 

Figure 1.„ Camera Location and Multiple Display Option 



SlDglaMsite - One approach to reducing the cost of 
the camera system is to display ail three images on a sin- 
gle, larger display, While the elimination of two displays 
reduces system cost, it does Ifttie to improve the drivers 
perception of the system, The driver must still integrate 
information from multiple windows in order to make a 
decision about the environment The use of a single dis- 
play may be detrimental to the ease of use of the system 
as flexibility in the location of the three camera images is 
lost 

MeawAiiU$S& - Another option is to merge the three 
camera images into a single integrated Image to be 
viewed on a single display. This approach is Illustrated in 
Figure 2, In this example, three camera images are used 
to synthesize a virtual image that shows the entire scene 
to the rear of the vehicle, In this synthesized image the 
redundant information from the three images is elimi- 
nated. In addition, information from ail three image 
sources is placed in the appropriate relative perspective 
providing drivers intuitive, visual information about their 
surroundings. 



Right 
Camera 




Merged Image 

Figure 2. Merged Image Example 

This architecture has the cost advantages of a single dis- 
play while providing two distinct advantages: 

* Full coverage of the rearward scene - eliminating any 
significant biSndspots and the requirement for drivers 
to direct their eyes to specific locations 

* integration of ail Information relating to the rear 
scene into a single image - eliminating the need to 
check and integrate multiple sources of information 

It is these advantages that more than compensate for the 
system's loss of depth cues and limited resolution, result- 
ing in a cost effective improvement to traditional rear view 
systems. 
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PERFORMANCE REQUIREMENTS - Given a specific 
architecture, there are two performance factors that sig- 
nificantly impact the cost effectiveness of a camera- 
based system: Field of View and Resolution. The affect 
of each of these factors is described in the following sec- 
tions, 

FieicLof View - One of the major contributors to safe driv- 
ing is the availability of visual information. The most 
important source of visual information in normal driving is 
within the driver's direct forward field of view. However, 
much important information is contained within the bal- 
ance of the driver's surroundings or secondary field, par- 
ticularly in multi-lane, high traffic density situations where 
lane maneuvers are frequent The goal of a rear vision 
system is to provide a field of view that contains all of the 
important information within the rear view. Typically mir- 
ror systems do not provide a field of view that includes all 
the areas of greatest importance within the rear field, 
resulting in well known "blind spots." The goal of any 
camera-based rear vision system should be to clearly 
improve the rear field of view beyond that available 
through conventional means, and should ideally include 
all the secondary field areas of greatest importance. 

Safe driving requires that maximum attention is paid to 
the forward view, and therefore information required from 
the rear view should be acquired in the shortest possible 
time. Several studies have focused on the time required 
to extract information from rear vision systems, in a test 
In which drivers were requested to make lane change 
maneuvers only when sufficient information had been 
acquired to ensure a safe maneuver, various rear vision 
systems were compared by measuring the time spent in 
acquiring rear view information and reporting it as a per- 
cent of maneuver time. This time included any head 
movement to acquire supplementary information that was 
not available through the mirror system. 1 

For a 32° field of view (FOV) inside mirror and 25° FOV 
left hand outside mirror, information acquisition took 
50.6% of maneuver time. For the above system with an 
additional right hand convex 25° FOV outside mirror the 
acquisition time dropped to 45.9%, For a 72° FOV peri- 
scope system the acquisition time reduced further to 
32.4%. We concluded that, in order to minimize acquisi- 
tion time, the secondary fields of view should be pre- 
sented in a single location, ideally as a continuous image, 
such that information may be obtained in a single rather 
than multiple glances. The single presentation also elim- 
inates the need to "piece together" images - possibly 
requiring additional confirming glances. 

Figure 3 shows the normal field of view of a pair of 
human eyes. In Figure 3, lines have been superimposed 
to represent typical positions of the major limits to a car 
driver's field of view such as the top of the windshield, the 
hood line, and the 'A' pillars. If we also consider the bifo- 
cal user, the most appropriate location for a single rear 
view image is directly ahead of the driver and just below 
the hood line such that the forward field of view is not 



impaired. It is desirable to size the rear view image such 
that it remains within the area normally viewed by both 
eyes. 
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Figure 3, Driver Field of View 

Resolution and Visual Acuity - Visual acuity may be sub- 
divided into resolution acuity and recognition acuity. In a 
rear vision system we are concerned with recognition 
acuity but it is useful to consider system requirements ini- 
tially based on resolution. The human eye is equipped 
with photoreceptors which subtend approximately 0.5 arc 
minutes. This sampling rate corresponds to an ability to 
resolve the viewed scene to about 1 arc minute in good 
lighting conditions. Figure 4 shows images pixilated to 
various degrees, changing the information content con- 
sistent with the view of objects at various distances, and 
then scaled to a constant pixel size, providing a realistic 
simulation of how objects are perceived at various dis- 
tances. Sufficient information is required to allow a driver 
to recognize and make good judgment about other road 
users at least at a distance which allows other road users 
take safe corrective action. A target recognition perfor- 
mance for a driver using a camera vision system can be 
determined based on a typical driver reaction time and 
stopping distance for a vehicle traveling at 160 km/hr 
(100 mph) and the ability to recognize a vehicfe when pix- 
ilated at approximately 100 mm square. This analysis 
leads us to a resolution design target of 6 arc minutes, or 
a camera sampling capability of 3 arc minutes. 

In order to optimize system performance, display resolu- 
tion should match the camera capability, and the display 
should be sized and positioned to match the human abil- 
ity to resolve the displayed pixilated image. In addition, 
the display size/resolution should provide image sizes 
that match the driver's expectation of relative object size 
vs, distance. We have established above that the camera 
should sample the object scene at a minimum of 3 arc 
minutes. It is assumed that, if cost is to be minimized, the 
display will be linked to the camera output with a pixel to 
pixel correspondence. In turn each pixel of the display 
should subtend no less than 1 arc minute of driver view to 
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avoid any system loss, and should probably match the 
sampled resolution of 3 arc minutes as closely as possi- 
ble to maintain distance perception cues. Thus, for exam- 
ple, If the displayed horizontal rearward field of view is 
6G*\ the display should contain 1200 pixels in each hori- 
zontal row, and If the display is positioned 80 cm from the 
driver's eyes, the horizontal dimension of the display 
should be no less than 20 cm. Note that this results in a 
practical 60 pixels per cm of display. 




Object at 100 m (0,08 miles) 



Figure 4. image Resolution Example 
P AST WORK 

Our first efforts in the application of cameras to automo- 
tive rear vision centered around the use of multiple cam- 
eras and displays to directly replace the exterior rear view 
mirrors. After a short period of experimentation we dis- 
covered the drawbacks of this approach and began the 
development of a system that would combine the three 
camera Images onto a single display. 

GOALS - The purpose behind developing the prototype 
system was to provide us with a platform that we could 
use to verify the utility of the thee camera, panoramic rear 
vision system. In addition, the development platform was 
to be used to experiment with the effect of various perfor- 
mance factors on the perceived quality and usefulness of 
the system . 

This being the case, practical considerations such as 
system size, power consumption., and maturity of appear- 
ance were not major design considerations. The devel- 
opment efforts focused instead on providing a high 



degree of modularity and flexibility in the nature and con- 
tent of the image displayed to the driver, in addition, we 
recognized that the electronics and display technology 
required to Implement a cost effective system would not 
be available for several years. This led us to the decision 
to apply more processing electronics to the problem than 
we would have had we been focused on developing a 
system that was nearer to production; 

DEVELOPMENT PLATFORM DESCRIPTION - A photo- 
graph of the development platform, including an overlay 
showing the location of the cameras and the field of view 
covered by each camera, is shown in Figure 5. As you 
can see from the picture, we selected a minivan as our 
development vehicle. This allowed us to implement a 
bulky system while locating the electronics within the 
interior of the vehicle and made wiring of the system sim - 
pler, 

A block diagram of the development platform is shown in 
Figure 6, The three cameras required for the panoramic 
vision are fed into a data processing module. In addition, 
a fourth sensor Is fed into the data processing module. 
This fourth sensor is used as a reversing aid and shows a 
field of view below what is visible looking through the rear 
windscreen from the driver's seat. 

In addition to the video images, vehicle status information 
such as speed and gear shift position are fed into the 
data processing module. The data processing module 
uses this information to control the Image processing cal- 
culations needed to combine the three images into a pan- 
oramic scene,. 




figure 5, Development Platform 



The system displayed the merged video on a 12.7 cm 
diagonal LCD mounted in front of the driver, just below 
his forward field of view during normal driving. In addition 
to the video, the system displayed rudimentary vehicle 
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statusinfbrmatbn such as turn signal and speed-informa- 
tion. 



through the vehicle as it both eased cable length restric- 
tions and reduced cable size. 



CURRENT WORK 

While the development platform was successful in dem- 
onstrating the feasibility and utility of a panoramic rear 
vision system. It did not do so using techniques consis- 
tent with production goals. Since the completion of the 
development platform, there have been significant 
increases in the performance and density of digital signal 
processing electronics, miniature CMOS cameras, and 
display technology. We felt that it was feasible to develop 
a prototype system that would closely resemble in form, 
fit, and function a production system, 

GOALS - The primary goal of our current activities is to 
leverage advances in technology to develop a panoramic 
rear vision system and integrate it into a test vehicle, in 
this effort, size, power and image quality are primary con- 
siderations, 

NEW SYSTEM DESCRIPTION in the development of 
the new system we set stringent power, size and weight 
requirements. This gave us some flexibility in the selec- 
tion of a test vehicle and we chose a sedan with conser- 
vative styling and a CAN network. The CAN network was 
used to transmit vehicle status information lo the elec- 
tronics as well as to provide for in-vehicte reprogramming 
of the system software. 
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Figure 6. Development Platform Block Diagram 

A block diagram of the system is shown m Figure 8, This 
system uses four cameras - three for the panoramic 
vision and a fourth as a reversing aid. The two cameras 
in the rear of the vehicle are connected to a small module 
that selects one of the video signals and transmits it to 
the main module over a high speed> serial link. This 
improved the flexibility we had in running the signals 
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Figure 7. Mew System Block Diagram 



The main electronics module, including the display, was 
packaged into the header console - replacing the inferior 
rear view mirror, We selected this location for two rea- 
sons: 

* This location minimizes the effect of the system on 
the interior styling of the vehicle 

- Drivers are trained to look to this location for informa- 
tion about the rear scene. 

While we are not certain that this is the best location for 
alii cases, we felt that it was worth evaluating. The follow- 
ing sections describe the major components in further 
detail, 

C.&in£M& - Our Initial demonstration activities had used 
CCD based cameras. While they were bulky and used a 
great deal of power, they offered excellent performance in 
terms of "resolution, low light level sensitivity and color. In 
the time since we completed our initial developments, 
CMOS camera technology has significantly advanced In 
terms of performance, 

CMOS camera technology offers a more integrated and 
lower power solution than CCD imagers, This results in 
lower cost and smaller cameras. In addition, the sensors 
are available with a simple digital Interface that reduces 
the size and complexity of the processing electronics.. 

Sased on these advantages we decided to develop cus- 
tom camera boards using VLSI Vision LTD's VV6404 
CMOS image sensor. The custom boards allowed us to 
develop extremely small cameras that met our interface 
requirements. 
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Display - Given our success with LCD technology in our 
earlier efforts and the general acceptance of this technol- 
ogy in the automotive market, we elected to continue to 
apply this technology. We were, however, able to lever- 
age two recent advancements. 

The first is the availability of wider aspect ratio displays. 
The panoramic video image shows a very wide field of 
view In the horizontal axis, but a relatively small field of in 
the vertical direction. This results in an image with a 
rather wide aspect ratio. 

Our first vehicle used a 3;4 aspect ratio display. When 
we mapped the panoramic image onto this display we 
were only able to use 50% of the display area for the 
video. Recently, displays have been released in a 9:16 
aspect ratio. Mapping our image onto this format 
resulted in our using 66% of the display for our video 
image - a significant improvement. 

The second improvement to the display was in the inter- 
face. At the time we selected our first display, smaller 
LCD's were only available with analog interfaces. As 
much of the image processing is performed digitally, this 
required a D/A conversion. Newer displays are available 
with a digital interface - reducing the complexity of the 
electronics and improving the noise performance of the 
system. 

In the end we selected a 18 cm diagonal, wide-VGA dis- 
play manufactured by Sharp, This provided the best fit of 
size, resolution, aspect ratio, and cost for our system. 

Electronics - It is in the electronics that the largest differ- 
ences between the old and new systems lie. The original 
system was required to deal with analog cameras and an 
analog display interface. In addition, the original system 
used analog video standards to transmit images from 
one sub-module to the next. As much of the image 
manipulation was performed digitally, this required multi- 
ple analog to digital and digital to analog conversions - 
increasing the complexity of the system while reducing 
the image quality. The new design offers the following 
features: 

• Completely digital system, from camera to display - 
improving image quality and reducing signal noise 

• Integrated, common processing for video manipula- 
tion and graphical overlays 

• Single-board design reducing system interconnect 
and size for increased reliability and lower package 
volume 

• Hardware-accelerated bilinear interpolation of pixel 
data for smooth scaling, stretching, and rotation of 
video data and other graphical images 

• Embedded 32-bit microprocessor for real-time con- 
trol of video processing and graphical overlay func- 
tions, 

A functional block diagram of the electronics is shown in 
Figure 8. The sensor data is written into a large texture 
memory under hardware control. A general purpose 



microprocessor then sets up block transforms from the 
texture memory into a frame buffer by setting the desired 
rotation, scaling, and translation coefficients. The hard- 
ware then performs the commanded transforms, forming 
a composite image in the frame buffer Once all of the 
required transforms are complete, the hardware then dis- 
plays the final image on the LCD. 

CURRENT DEVELOPMENT STATUS - We have recently 
completed the development of our new prototype pan- 
oramic rear vision system and installed prototypes in sev- 
eral vehicles for further technical evaluation. We are 
working with several automotive manufacturers to deter- 
mine technical, cost, and schedule requirements for 
potential production opportunities, 

SYSTEM PERFORMANCE - One of the goals of this 
activity was to demonstrate an improved performance as 
a result in technical developments. The following table 
summarizes the technical performance of the system: 

Table 2. System Performance Parameters 



Parameter 


System Performance 


Screen Aspect Ratio 


9:16 


Effective Screen Size 


5.3 cm X 16 cm 


Effective Screen Resolution 


320X854 


SNR 


32 dB 


Graphics 


Hardware Accelerated 


Update Rate 


30 Hz 


Latency 


60 ms 


Electronics Volume 


300 cm 3 



FUTURE WORK 

While our prototype has been successful in demonstrat- 
ing the feasibility and utility of panoramic rear vision, the 
development activities are far from complete. In order to 
successfully sell this concept into production we will need 
to address both the cost effectiveness of the system as 
well as redesign the electronics for production. The 
issues relating to each of these tasks are described in 
the following sections. 
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Figure 8. Electronics Block Diagram 
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ADDITIONAL FEATURES -Displaying full motion color 
video required the use of a relatively expensive display 
technology. The advantage of this display technology is 
that it can be adapted to multiple functions to improve the 
cost effectiveness of the system. In addition to displaying 
panoramic video images, the display can be used to 
present vehicle status or navigation information to the 
driver 

This is where the decision to use a common graphics/ 
video processing architecture begins to have advan- 
tages. The same hardware we used to perform the video 
processing can be used to accelerate texture-mapped 
graphical representations ranging from simple instrumen- 
tation to complex moving maps. 

Future development efforts will focus on using this capa- 
bility to add features to the rear vision system. We will 
begin by adding the functionality associated with a tradi- 
tional instrumentation cluster and continue by developing 
context-sensitive formats for display of vehicle status and 
control of other vehicle sub-systems. By leveraging the 
multi-functional nature of the display we can use the sys- 
tem to address a variety of control and informational 
needs. 

PRODUCTION ISSUES -As stated earlier, the proto- 
type was not developed with the intent to transition it 
directly to production. Our efforts to turn this into a com- 
mercially viable product will focus on a combination of 
performance improvement and cost reduction. 

The implementation of the electronics is one area where 
we will focus on cost reduction as well as improving EMI 
and EMC compliance. The custom digital logic required 
to accelerate the texture calculations needed to merge 
the video was implemented using relatively expensive 
memory technology and a reconfigureable part. Signifi- 
cant cost savings will be achieved by implementing the 
logic in a custom component and by using a more cost 
effective memory technology. 

Progress can also be made in improving the performance 
of the camera. While CMOS camera technology is inher- 
ently lower cost than the alternative and has several per- 
formance advantages in terms of blooming, its noise 
performance - particularly at night needs to be improved. 
Future generations of CMOS cameras will implement fea- 
tures such as micro-lensing to improve the low light level 
sensitivity. 

A final area where we plan to make progress is in the 
optics. We are currently investing in the development of 
low cost plastic optics. This new technology allows the 
cost effective implementation of lenses that use a combi- 
nation of diffractive and refractive elements that are quite 
cost effective and low weight. 



CONCLUSION 

We believe the future of automotive information systems 
lies in the successful integration of multiple sources of 
information for presentation to the driver with the intent of 
improving safety and reducing work load. We have suc- 
cessfully demonstrated applying this concept to the rear 
vision system - replacing three mirrors with a single inte- 
grated display. 

Daily advances in electronics and display technology 
continue to improve the cost effectiveness of these sys- 
tems. We believe that it is a matter of when, as opposed 
to if, electronic information systems will become com- 
monplace in automotive applications. 
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Looking Back to the Future - How hard can it be to eliminate a 
driver's blindspot? 

TOM MURPHY 

Ward's AutoWorld, May 1, 1998 12:00 PM 

Automotive innovators have tackled plenty of safety issues over the years from softening 
the impact of collisions to providing greater control during emergency braking. A 
satellite high above the Earth can pinpoint, within a matter of meters, the location of a 
stranded moton st. 

Still, unless a driver turns his head at least 90 degrees, he cannot tell with any certainty 
whether a vehicle is in an adjacent lane. It surely contributes to road rage, as well as 
potential accidents. In one study, the National Highway Traffic Safety Admin. (NHTSA) 
estimates lane change and merging accidents account for 16% of all major collisions. 

Today, suppliers are pursuing numerous technologies to eliminate the pesky blind-spot, 
from more easily adaptable asphenc side mirrors to advanced - and more expensive - 
cameras and sensors designed to detect any object behind or beside a moving vehicle. 

The technologies are not terribly new, but they have been refined over the years - and 
costs are coming down. In addition, NHTSA, which allows only passenger side mirrors to 
be convex or asphenc, now considers allowing asphenc mirrors on the driver side. 

Convex mirrors are bent, making objects appear smaller and farther away. Asphenc 
mirrors are flat, but the outer edge is angled outward. Both mirrors significantly expand a 
driver's field of vision. Convex mirrors cost slightly more than flat ones, while asphenc 
mirrors can be several times more expensive, depending on the curvature. 

A Dutch research group is preparing a report for NHTSA, due this June, on how 
European drivers interact with aspheric and convex mirrors, which are used on both sides 
of vehicles in that region, says Stephen Kratzke, director of NHTSA' s' Office of Crash 
Avoidance Standards. 

"If we get information that they (asphenc mirrors) will work, then we would allow them," 
Mr. Kratzke says. "We want to know if people will catch on quickly to how you use them 
to expand your field of view." 

NHTSA doesn^t take this stuff lightly. The agency currently requires that driver-side 
mirrors be flat, partially because motorists complained of headaches when NHTSA first 



allowed convex passenger-side mirrors in 1983. Mr. Kratzke says it's been a long time 
since NHTS A heard such complaints. 

Suppliers are not counting solely on aspheric mirrors as the future in rear- vision 
automotive applications in the U.S. Many are investing in sensors and digital camera 
technology to tell a driver whether an adjacent lane is clear. 

Donnelly Corp., which makes 95% of the prismatic mirrors used on U.S. cars today, is 
delivering a three-camera "panoramic vision" system to replace the mirrors in a concept 
vehicle being produced by a customer within the next year. 

The system replaces the side mirrors with tiny digital cameras, about the size of a golf 
ball, and adds a third camera in the rear of the vehicle. All three images are converged 
into one, which can be displayed for the driver on a screen on the instrument panel. 

The system provides full vision of everything behind and beside a moving vehicle. It 
nullifies the need to turn your head, for those people who don't trust mirrors. 

Donnelly developed the system with Vision Group pic, its partner in Edinburgh, Scotland. 
Vision Group provides the CMOS imaging chip driving the system. 

Are drivers ready for it? "We have the functionality," says Ken Schofield, Don-nelly's 
executive director of advanced engineering. "The question is price and application. It 
requires a change on the part of drivers." 

If NHTS A struggled with convex mirrors, it surely will view camera vision with extreme 
caution because of the potential distractions for drivers. 

"Camera vision is like having a TV on, to the extent that it's taking the driver's attention 
off the road," Mr. Kratzke says. Still, he concedes, drivers already are looking at mirrors 
in three different places. Combining the images into one central location could be an 
improvement. 

But government acceptance isn't the only hurdle. Cost is a major consideration. Mr. 
Scho-field estimates the pricetag could be $1,000, going down to $350. Eventually, costs 
could be comparable to existing mirrors, depending on the features and configurations. 

"It's just too expensive," says Ken LaGrand, executive vice president for mirror supplier 
Gentex Corp., which is developing rear-vision systems with its partner, Photobit Corp. of 
Pasadena, CA. "As a consumer, I would like it on my vehicle, but my feeling is it's 
probably got to come down in price to a few hundred dollars. That might be a little 
optimistic," he adds. 

Mr. LaGrand says it will be several years before a U.S. vehicle is equipped with rear- 
vision cameras. Meantime, Gentex is pursuing aspheric mirrors, which started out in 



Scandinavia 10 years ago and are expected to outnumber flat and convex mirrors in 
Europe within the next few years. 

Gentex recently announced it will supply the world's first aspheric exterior electro- 
chromic, or automatic-dimming, mirrors for several Mercedes-Benz luxury sedans. 

Another method of eliminating blindspots is with sensors that beep to warn drivers that 
the lane they are moving into is occupied. Several suppliers, including Donnelly and 
Gentex, are developing such systems, 

Autosense Ltd. of Denver, CO, has teamed with Siemens Components Inc. to develop 
SideMinder, which uses sensors located near the rear taillights to alert a driver, with a 
flashing warning light, if a vehicle is in the right or left blindspot. 

Alirt Advanced Technology Products, a small Canadian company, is planning 
commercial delivery of a battery-operated aftermarket blindspot sensor this year. 
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COMPANY 



Donnelly Corporation is a technology driven, customer focused auiomodve 
supplier known for offering the most advanced technologies for automotive 
mirror systems, window systems and electronically sophisticated sensor systems 
to customers around the world, Donnelly and its affiliates employ more than 
7,000 people- in the U.S. and twelve other countries in North America, South 
America, Europe and Asia. 
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r Buzz? . . . Electronics 

During 1999 Donnelly 
won a major new business 
commitment from 
General Mourn m build 
interior mirrors that 
contain, advanced 
electronics to support GM's 
.OxiStar® consumer 

relea>mmumearions service. The OnStar® 
package is a good example of bow 
automakers axe beginning to view mirrors 
as an attractive place ro locate electronics, 

But there's more ro Donnelly's 
electronics story than mirrors, in £aci:> over 
the long term were hoping to use 
electronics to make out own mirror 
products obsolete. That isn't heresy, its just 
smart: thinking. 

Picture This 

For nearly a decade, 
Donnelly has been the 
pioneering leader in the 
development of camera vision 
systems that ultimately can 
replace mirrors on cars and 
light trucks. Why? Because we 
believe camera vision offers a 
better approach. Camera vision 
eliminates blind spots and trees 
automotive designers from the 
need to incorporate bulky exterior 
mirror housings on vehicles of the 
future, That dramatically reduces 
wind resistance and improves fuel economy. 

Late in 1999> GM introduced its ultra 
high-mileage Precept concept vehicle, which 
included a .sophisticated. Donnelly camera 
vision system. Donnelly s unique system is 
built around proprietary software that 




4 hands-free microphone and 
convenient push buttons mounted in the 
interim mirror wiiigive drivers easy 
access to the OnStot® roadside 
assistance caii £$nt$r, Donnelly will 
start production of these advanced 
mirrors during the summer of 2000, 
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R&placing outside mirrors with 
smaller, more aerodynamic camera 
pods can mi wind drag by 15%! 



merges the digiril -images from three 
or. more separate cameras into a single 
panoramic image. Through that image, 
drivers have a dear, unified view of the 
fear and -skies of the vehicle. 

By eliminating the need for outside 
mirrors on die Precept? GM engineers 
estimate they were able to reduce die 
vehicles drag by- about 35 percent. 
That's better than getting your 
overweight brothef>in-kw out of 
the back seat! 

in addition to the camera vision 
system, Donnelly extended its 
technologies a step further on 
the Precept program. Built inco 
the instrument panel of the 



vehide is a single-screen display that shows 
the camera vision image and other essentia! 
vehide information—such as speed , fuel 
levels and other data— thai: traditionally have 
been shown on separate gauges on the 
instrument panel. Donnelly designed and 
developed the central display, which provides 
an easy-to-read information source and helps 
reduce overall vehicle weight. 

The .Precept system was just one of the 
camera vision systems Donnelly delivered last 
year to automakers in North America, 
Europe and Asia. While full camera vision 
systems on production vehicles may still be a 

few years away> we expect that 
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GM Precept: Concept Vehicle with Lowest Drag 

2000,01.05. 

DETROIT - Opel's parent company General Motors will unveil the Precept a fully 
functional hybrid electric vehicle with the lowest drag coefficient (C d = 0.163) 
ever recorded for a five-passenger, four-door family sedan, at this year's Detroit 
Motor Show (January 15 - 23). 

Parallel hybrid drive - Advanced battery systems - Extensive use of aluminum 

The Precept is powered by an electric motor driving the front wheels and a turbo 
diesel engine with common-rail direct-injection driving the rear wheels. The 
parallel-hybrid propulsion system, managed by sophisticated electronic controls, 
helps the Precept approach its target fuel consumption of 80 miles per gallon 
( 2.94 liters per 1 00 kilometers ). 

Two of the world's most advanced battery systems are being explored for use in 
the Precept - the world's first application of lithium polymer batteries for hybrid 
vehicle propulsion, and a new generation of compact nickel metal hydride 
batteries. Aluminum is used extensively throughout the vehicle's body and 
chassis to reduce mass. 

The Precept is the latest demonstration of GM's capability and commitment to 
developing efficient and affordable environmentally compatible vehicles. It 
evolved as a result of GM's involvement in the Partnership for a New Generation 
of Vehicles (PNGV), a joint effort between the US Government and the US auto 
industry that began in 1 993. The specific aims of this partnership are lower 
emissions and significantly improved fuel efficiency without compromising safety, 
performance, affordability, or utility. 

Whereas the PNGV parameters were the catalyst for Precept, the high-efficiency 
architecture and the propulsion system are a unique solution created by GM. 
Moreover, the same technologies developed in the Precept program have been 
leveraged by other advanced vehicles at GM, including the high-efficiency, low- 
emissions Opel G90 shown at the Frankfurt Motor Show last September. 

Precept Feature Highlights: Technologies of the Future Today 

The dual-axle regenerative, parallel hybrid propulsion system of the Precept 
features a 35 kW three-phase electric motor driving the front wheels and a three- 



cylinder, 

1 .3-liter turbo diesel engine (40 kW/54 hp) with common-rail direct-injection 
driving the rear wheels. Multiple energy-conversion devices are used for 
propulsion because that facilitates operating each in its zone of optimum 
efficiency. Sophisticated electronic controls interpret the driver's commands and 
manage every aspect of the propulsion system's operation. 

The brain of the entire hybrid propulsion system is a PC processor-based 
controller. This 32-bit, 266 MHz device conducts a two-way dialogue with the 
following equipment: accelerator pedal, brake pedal, gear shifter, energy 
management system, electric traction system, multi-purpose unit, diesel engine 
controller, transaxle controller, thermal system controller, and brake controller. It 
makes snap decisions to optimize all operations, such as the highly intelligent 
heating and cooling system. Specifically, the system moves otherwise wasted 
thermal energy automatically to where it's needed - it even anticipates when fog 
could form on the windshield and works to prevent it. Throughout the Precept, 
forty-seven distributed computer modules provide smart control to make this the 
world's most computerized vehicle. 

Key Component: Computer-Controlled Automatic Gear Shifting 

A key component of the hybrid powertrain is a unique, high-efficiency, 
automatically shifted manual transaxle. Fifth and reverse gears were eliminated 
(reverse operation is provided by the electric traction system in the front of the 
vehicle). Gear shifting via clutch was converted to an automatic, computer- 
controlled operation. 

As a result, the diesel engine can be started in any gear, and skip shifting is also 
possible. In addition, the time required for a gear change, or to start the engine, 
is about 0.7 seconds. Most importantly, efficiency losses through this transaxle 
are particularly low. 

Another key benefit of the dual-axle, parallel-hybrid approach is that it offers 
maximum use of regenerative braking through all four wheels. When the driver 
lifts off the accelerator or taps the brake pedal, unwanted momentum is 
automatically converted to electrical energy and used to recharge the battery 
pack located under the front seat. That alone amounts to more than 0.5 1/100 km 
in combined city-highway driving, helping the Precept approach its target of 2.94 
1/100 km. An ingenious multi-purpose unit handles regenerative braking through 
the rear wheels in addition to performing five other functions: it contributes 10 kW 
of propulsion power when maximum acceleration is desired, it starts the rear- 
mounted engine, it powers the air-conditioning compressor when the engine is at 
rest, the engine can drive it as a generator to recharge the battery and it is used 
to synchronize the gears for clutchless shifting. 

Architecture: Lightweight and Ultra-Efficient 



The Precept uses a light, stiff space-frame body structure constructed of 
aluminum stampings, extrusions and castings. Exterior panels are made of 
aluminum and composite materials. Bumper beams are fabricated with carbon 
fiber. 

Since the weight carried by the front tires has been significantly reduced, no 
power steering is necessary. Computer-controlled air springs maintain a level 
ride height irrespective of passenger and cargo load changes. The springs' 
source of compressed air can be tapped to inflate the Michelin Proxima low 
rolling-resistance radial tires. The Precept's aluminum wheels each weigh only 
3.8 kg , distinguishing them as the world's lightest 16-inch wheels. The friction 
brakes for the Precept save even more weight. An advanced brake-by-wire 
system uses a small electrically powered hydraulic pump located in close 
proximity to each brake caliper. 

Aerodynamics Record: World's Best Four-Door Drag Coefficient 

Maintaining a controlled ground clearance in conjunction with the use of a flat 
underbody, rear cooling air entries and exits, and other aerodynamic features - 
along with extensive wind-tunnel tuning - has resulted in the lowest drag 
coefficient ever recorded for a five-passenger, four-door family sedan (C d = 
0.163). So efficient is the Precept's shape that just the addition of conventional 
outside rear-view mirrors would increase drag by more than 17 percent. 
Consequently, the Precept employs two tiny cameras instead of outside rear- 
view mirrors which, when combined with a third rear-facing camera located inside 
the rear window, produce an integrated panoramic rear view on a reconfigurable 
LCD display easily viewed by the driver. 

Interior: Attractive, Functional and Driver-Oriented 

The Precept's interior is nearly as futuristic as its exterior and mechanicals. 
Advanced interior trim materials not only save additional weight, they also 
provide added convenience and an inviting appearance. The ultra-light front 
seats consist of powder-coated steel and aluminum frames with a bare minimum 
of foam padding and trim materials. A Driver Control Center provides a unique 
driver-vehicle interface used to operate the car. To start and drive the car, no 
mechanical key is needed. Instead, the driver enters a five digit security code 
using the electronic key pad positioned below the display screen, hits the RUN 
button, and shifts the vehicle out of park and into forward or reverse gear using 
another set of push buttons. 

A smart switch in the control panel operates like a mouse for the laptop computer 
docked at the top of the instrument panel. It is used to scroll through a menu 
displayed on the laptop screen. The screen is positioned far forward just under 
the windshield's base, so minimal refocusing is necessary when shifting between 
a view of the road and a check of the instruments. Three distinct categories of 



information are presented: a view to the rear supplied by the three video 
cameras; vehicle operating data, such as speed and fuel remaining; and 
secondary entertainment and climate control system parameters. 
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Donnelly PanoramicVision on Renault Talisman Concept 

Car At Frankfurt 

BgOgyiiVgLSgfe w iH a ' so feature Donnelly interior electrochromic 
rearview mirror and exterior electrochromic powerfold mirrors. 

FRANKFURT, Germany . Sept. 10 2001 

Qgn^[!^^L^Igl^. and Rgggujt have teamed up to demonstrate new technology at the 
2001 j^nMj^Mg!^^^ September 11-23. Donnelly is supplying enhanced 
driver visibility and safety products on vehicles. 

Donnelly's next generation Pa nora mjcVj sjon (T M ) system is demonstrated on 
the SSiOMl "'IMSIll^Q 11 concept car. There are no mirrors on the car. 
Instead, this system digitally merges the images from three cameras to relay a 
seamless panoramic view of the area behind and around the vehicle. The view 
is displayed on a screen located on the upper part of the dashboard. The 
screen also provides information on navigation, the sound system, the heating 
and cooling system, and the vehicle's warning and security system. 

In addition, Doexi^ has other advanced products on Renault's production 
vehicles featured at the show. Renault's Ye I Satis luxury car is equipped 
with a Dgineljy interior auto-dimming electrochromic IrearvjewMrroEi which 
houses both a humidity and rain sensor. This new luxury car also features 
Donnelly's auto-dimming electrochromic outside mirrors with a powerfold 
feature that allows the vehicle to be parked safely in tight spaces. 
Renault's Avantime, displayed at the show, will also feature Donnelly interior 
^3-dimming rearview mirrors. 

"Donneyv is very pleased to work with Renault, to demonstrate our 
leadership in vision technology," said R^n|^ vice president, 
Cgj^orat^^ ''Our technology will bring an improved level of safety 

and styling to the vehicles of tomorrow." 



